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Abstract • ' , 

L ' • 

.The effects of tm data-utilization rules on spelling achievement 

■ <► • - ■ 

were^ compared for an* 11-yeai^ ora. boy^who. had^^been diagnosed as 
learning disabl^ed. During instructional Sessions, the h6y was taught 



and measure^ on 'sets of difficult spelling words. Graphed data were 
analyzed using a concun;ent schedule design whereby equivalent 
behaviors are treated simultaneously with ■ different approaches ' to 
determine rela.^ve treatment effects. One treatment approach involved 
the • following data-util i^tion rule: If the student's performance 
tell below the- expected lev^l on three consecirtive days, ^the teacher 
introduced a^ program change. In the' second treatment,' the teacher 

•made changes 'in the student's program every 5 to 10 v^ays. Results 
indicated that, 'in the se'cond data^ti 1 izati.on condition, \he 
student's trend lines over/ a six-week trea'€ment period were superior 
to trend lines in* the first data-utilization condition. Implications 

"for practice and further research needs are discussed. 



* • " The Effect of Alternative Data-Utilization Ru-les on 
Spelling Achievement: An N of 1 Study ^ 

■■ . > ■ ■ . • ■ " • ^ ^ 

* PL 94-142 requires special edncators not only to develop an 

individual educational -^program (lEP) for each identified handicap.jed 

student, but also to monitor the lEP objectives and to make* an effort 

to aS'Sist each child in aoHreving. [lis/her goals. Currently, special 

educators typically monitor> progress toward goals in an unsystematic 

^and informaj^ fashion, . employing summative evaluation to certify 

program completion and to attest to the amount ofv progress made. This 

evaluation technique is .an after-the-fact format whereby inadequate 

progress is documented rather than acted upon. ' ^ 

* . • 

An alternative evaluation procedure is formative; frequent and, 
direct informati-on on stl^dent progress is collected to determine 
regularly the- effects that program changes have on pupi^ performance. 
Wtth formative evaluation, teachers can make changes 'as^ necessa^^y to 
^assist 'children in meeting goals (Starlin, 1971). Although formative 
evaluation appdars to be more effective than summative evaluation in 
facilitating children's goal attainment (Hari'ng, Maddox, & Krug, 1972; 
Haring -& Krug, 1975; Jenkins, \Mayhall , P^chka, .& Townsend, 1974), 
inadequate informatitjn exists to determine which aspects |^f formative 
evaluation are essential- in producing those different^l effects. 

The effectiveness of data-utilization rules is an aspect of 
/formative evaluation that" has been investigated. ' "With datj..- 
utilization ruleS, tecichers are required to implement instructional 
changes whep^student performance data conform to a prespecified 
pattern. Occasional studies of the effects of alternative data uses 



indicate the 'potenYal importance of employing such prespecif ied 
..rules.. Frum^ss .(1973) randomly selected 45 boys from 15 self- 
contained classrooms., for minimally brain-injured (MBI) students and 
assigned each , to ^ one of ^ five conditions: (1) Self-Chart , Self-Set 
Aims (SCSSA), where ' students graded, • tal 1 ied, and recorded their own- 
math fact performanGe and .set their own weekly aims on"graphs; (2) 
Self-Chart, Teacher-Set Aims (SCTSA)., where teachers set the weekly 
'aims; (3) Teacher Chart, Teacher Set Aims (TCTSA) where the children 
graded;;and tallied the'numtjer of math facts correct and incorrect per 
minut.e, but teachers charted and 'set' weekly aims.; (4) No Charting or 
.SettVnq Aims (NCSA) where children g^-aded and tallied but there was- no 
charting or aim setting;; and (s / Contro^^Group . The resuUpof this 
experiment are difficult' to interpret, since although the SCSSA and- 
" SCTSA groups made._ significant, gains over all oth^i^^^^r^s, tlie NCSA 
; group improved'^ significantly more than the -TCTSA group. . . Frumess 
speculated that teachers may. not have used the data the^' charted to 
implement instructiooaV changes. Such^an hypothesis supports the use 
of data-utilization rule^ to help teachers- realize, when ongoing 
programs ar^e inadequate^ and |hen instructional chjinges are tieces^sary 
to effec^t student growth. ^ f 

^ . In concert with this hypothesis, Wh.ite (1971) demonstrated that- 
teachers do not utilize data to make program changes unless they are 
required to do so. He- found that many teachers allow programs tp 

I ' r 

remain intact long after thos'e^ program's appear . to affect student 
progress. * ' 

In^an attempt to i-w4Kigate more directly the effect of data- 
utilization rules on children's academic performance, Bohannon (1975) 

\ 



compared student "achievement when teachers us.ed decisiort rules with 

'achievement when teachers relied on their clinical judgrnent. Teachers 

assigned to the first group employe^ the fol lowing^ decision rul.e': a 

child's program would be altered If his/her performance fell below a 

minimum daily expectation for two ^consecutive days. Results revealed 

that gains for the data-utilization rule group were thr^e to four 

times, greater than the average gain for the clinical judgment group.. * 

With ^elementary age students from special education resource 

programs,* Mirkin (1978) also studied the effects .of employing a data- 

utxlization rule on puprtls' reading-. performance. In this study, the 

decision rule appeared - to be the most important component of the 

formative evaluation system; fiv^ out of the six treatment differences 

•involved the d.ata-uti'lizat idn rule. Therefore j^-'evilde^ indicates 

that the use of *dat^-util ization ^rulps represent^^^^one component of 

formative evaluation, that accounts for improved student gains. 

Nevertheless, it remains unclear whicl^ specific rules are most useful. 

. . ' \^ 

The purpose of the present study, then, was 'to compare the 
. . s ' * * 

usefulness of different decisi^on rules. Spec>4fTcally, the effects of 

■ ♦ ■ ' \ . 

two data-utilization, rules on a student's spelling achievement were 

cclfiripared. The first decision rule is similar to that proposed by 

Liberty (19>2, 1975). It' specifies that a "static*' aim (final 

perform'ance objective) be specified, and a "dynamic" aim -be drawn on 

the graph .by connectij]^ the median baseline peiji^rmance with the 

static aim (see Figure 1). Then, the rule dictates that a change in 

the student's program be implemented whenever^ student performance 

falls' below the^^dynamic aim for three consecutive days. The use of 



thi!^ three-day decision rule (White & Haring, I976f js ba*s£d on the 
assumption ' that it is crit.ical tq alter ineffective strategies as 
early as possible to maximize the growth of special education students 
(Howell, KapTan, &0»Connell, 1979).. 



Insert Figure 1 about h^re 



The^ competing decision rule, investigatefi in the" current ' study 
relies on the assumption that, in addition to maximizing rate of goal 
attainment,/ it is never 'appropriate to assume that a child is 
achieving maximally. Therefore, regard/Tess ot^ whether student 
performance is commensurate with the expected . levelS of achievement; 
the special educator must make lyogram adjustments regularly to 
determine whether even greater progress i'i^ possible. This decision 
rule stems from an applied ^behavior analysis (ABA) framework, v^here 
the teacher is viewed as experimenter. Each intervention period is 
evaluated relativ.e to the effectiveness of previous ones. Effect^ive 
changes are maintai'ned or enhanced and ineffective procedures are 
dropped. In tJii.s conditJ^on," the ^decision rule was (a) 'to intj^oduce a 
program change and evaluate its effects every five to tfenrdays, (b) to 
make ^ -program change as soon after day fotir as the current program 
appeared unsuccessful., and (c) if the program ran a full ten days, 
make an adjustment even .if. the current program appeared relatively 
^successful. ^ ^ ' ' ' -T ' 
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1^ ' Method " ^ V 



Subject and Setting \ 

. The' student" who served as ~a ^subject «irt this stud;i?'was an 11 year 
old male fifth-grader who had been diagnosed as learning disabled. He 
_ attended/'a program for ^:hildren wiib^special learning and;, behav.ior 
problems (SLBP) that served approximately two percent'of children in 
grades one Ahrough siv-4n an upper middle-class midw^stern schoo'-l^ 
The students in this SLBP program left their 'regular classrooms, went 

. • , ^ 

"to a resource room, 'Snd spent frqm 30 mintites to 2 hours ,dai ly in 
smalK instructional groups. . Typically,-^ the average fifth gr|ider in 
this school scored two Vears above „grade- level on nationally 
standardized tests. Fifth grade students Enrolled in the school's 
S,LBP program scored s|x m6nths to one year below grade level on ' 



nationally standardized tests, 
r ^ Th^^ subject 'was-* selected 



because of his difficulty in learning 



spelling words. He was diagnosed as thaving above average learning- 
potential with poor achievement in all basic skill areas? The subject 
left his regular . classroom and spent <^ne hour per day in the school* s 
SLBP resource room, grouped with* a fourjh ' grade boy and a fifth gradfe.' 
grade boy/ He received assistanjpe vr\ reading, language arts, and 
math. ' ^ ^ ]i ^ 

Procedure " ' <> . \ 

Design. A co'ncui^rent schedule design (Hersen & Barlow,' 1976) was 
used to^lxamine the expertmental question of which decisioFi rule would , 
effect greater spelling, achievement.* In this experimental design, two 
equivalent* b^ehaviors are treated simultaneously with J different . 
approaches to determine relative treatment effects. 

. • • 10 \ 



. ■ • ^ . r 

- ^ Materials . ^ Two hundred spelltng "demons," or difficult spelling 

words, from Dr. SpeTIo (IC<l)ttmeyer , 1968) and;100 words from Teaching 

Children ,with Learning and Behavior Problems (Hammillv & Bartel, 1975) 

were transcribed^onto 3" ^'by 'S*' cards and randomly divided' irrto two 

word packs labeled Word Pack A and -Word- Pack" B. • The two word "packs 

wehe^assumed to' be equivalent in difficulty.. . ' 

Goal setting, measurement task, and .graphing . For the purpose, of 
determining ah- . appropriate goal, a' fifth grade xlassroom teac'her 
selected two girls and three boys from her middle spelling group. 
Children were tested indiv.idual ly. For one minute they wrote dictated 
words drawn alternately' frjom the two word packs. The ''"'medi an 
performance of these children was seven correct and four errors per 
minute; this leveT of performance Wcfe estabi ished as the subject's 
longy^erm goal. - * > ^ 

Throughout the study, the speWing. ..test or measurement task was 
an analogous onerminffte timing of the subject's writing randomly 
selected words from a word pac^k'. Words correct and errors per minute 
were scored. One graph was designated Chart A for Word Pack A and one 
graph was designated Chart Graph B for .Word Pack B. A static aim was - 
drawrt-^on each^c^iart. (iee Figures 1 ^nd 2 . ) - 



Insert Figure 2 about here 



v 

Baseline ; For seven da^, the student was tested on each word 
pack. The ^bj^ct's^ baseline performance, ' the medium score for the 
seven da^s, was 2.5 words correct per minute. A dynamic a.imline then 
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was drawn on each cha^^t from 2.5 at the end of ba'seline to the stattc> 

aim. (See Figures 1 and 2.) • 

y Treatmehts , , At the beginning of the study; the student recaived \ 

^five 'minutes of daily .dir^ect . instruction' on-a random selection of 

~ ( , 

words, from each word pack. Under one .condition', in :Chart A, the 

; ' • o • . ' ' 

decision rule iwas to make a strategy char\ge if the subject'^ - 

performance fell t?e'low Ms -dynamic -aimTTn^hree days in a row.- Under 

the second ' condition, in ' Chart B, the decision rule was to make a^ 

strategy'change every five to ten days, after four ta nine days if the^ 

intervention was ineffective, but;- no latemhan after^ten days even ;f 

the prb^gham appeared to be effective. 

Throughout the. study, the dependent data were the number of - 

correctly spelled words per minute and the number of incorrectly 

spelled words per ^'mi^nuter The SLBP teacher carried out the^various 

teaching strategies with the subject for about a.lO-minute period for 

eacb Word pack. She then shuffled- each 3'' by 5" deck of words five 

times and^ with' a stopwatch' dictated words 'from the deck for 60 

' seconds, V as the subject wrote them on paper. At the e^ of t'he^ 

measurement - period, the teacher counted words correct and incorrect 

from the deck and recorded the scores on the appropriate graph. On 

\. • 

alternate d.ays, cartii^ack A -or B was worked on and dictated first. 

Results \ ^ s . 

On Figure 1, ::he dependent data for condition A were graphed: 
spelling words 'correct and erro/^s per* minute per day under the 
decision rule whereby a change was- made When performance fe^] below 
the dynanic aim on three consecutive days. On figure 2, depaodent 



12 



ERIC 



data for conditipn B were graphed: ^spelling words correct and' errors 
per-miBUte per day under 'the decision rule vy;iereby changes were made 
.every five to ten days. \ ' . ^ * . 

"An analysis of 'these graphs confirms that baseline performante 
ran for seven days in both conditions and median performance for both 
condition' A and condition B was 2.5 words correct with seven errors 
per minute. Over the last' seven days of the experiment, performance 
in:,'Condi tion A was a median of five words correct and four errors per 
minute.-. Median performance in condition B over the last seven days 
was seven words correct and three errors per minute. A total of three 
program chang&s was .mcide in condition B, -v/hereas one program change 

was made in the other condition. . ^ 

~ ' • V < 

Trend lines^^^^lc-ulated by the split median method,, ranging from 
the data point on day '8 to the last, data point were drawn for 
conditions A.and B and are displayed in Figures 3 and 4.' To 'calculate 
the average ' r^te^of progress per day over each trend line, the 
absolute value of the difference between the data point along the 
trend line for d.ay ^ and -for day 33 was divided, by 25 days. In 
ond.ition A, the student made 'no average increase i.n words correct per 
minute per day arfd njade alj average decrease in errors per minute per 
day of .06. I-n condiVh^n B, the student made an average increase of 
.16 'words correct per minute per day and made an average decrease in 
errors per minute per day of .12. The average decrease for errors per> 
minute per day for condition 8 j^as two times greater than for £hat of 
condition A. Since the trend line » was flat -for words correct per 

^ i * 

minute per day for condition A, no discrepancy r?itio can be calculated 
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for , the, difference in trend lines forewords correct per .minute per 
day. , ^. * . • 



Insert Figures 3 and 4 about here 



Discussion . 
Although the student approximately met his goal - both 

s ^ ' 

conditions, he made superior progress in both wfJrds correct per minute 
per day and errors per minute per day in conditi-on B. Close 
inspection of Figures 1 and 2, reveals tl\at the student's performance 
'Sharply fell after baseline in condition B,/V(hereas it sharp-^y rose in 
condition A. In part;, this ^ explains how superior progress ^was 
manifested in condition B ev^n^^'tFiough the absolt!ite difference in final 
, performance between the two conditions was negligible. At about day 
8, the student expressed the opinion that Word Pack B . was more 
difficult than Word Pack A. However, as performance began to climb in 
condition B, the student expressed the belief that the two' word packs 
were similar in difficulty. Therefore, it appears that the student's 
perception of word difficulty was influenced by -his. performance leVief^l"^ 
and in no^ way explains the discrepancy between the post-baseline 
levels of performance on the two charts. 




I Xjl pf the program adjustments except the last one in condition B/ 
wer.^ instructional changes incorporating first oral then written 
practice. These types of interventiq'ns proved to be effective. This 
effectiveness is highlighted by the sharp increase reflected in the 
last three data points for words Correct per minute in condition A; 
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these" data points were obtained, following the introduction of^^l-^nd 



written, practice. The trend had been flat before the introduction of 
this last change. The student appeared to be putting forth his best 
effort;, so the 'motivation' intenvention introduced as the lasft .ch-ange 
in condition B did not produce .a dramatic shift in trend. 

A positive practice session seemed to produce better results 
during that day ' s measurement session in both conditions A and B.. The 
student had a measurement) session on each word pack each day, 
. alter/iating packs: on which' he ywas measured first. • The ,,;)evel of 
performance on the first word pack appeared to affect the level of 
performance "gri the second word pack. Therefor(e, there apparently was 
some transfer in performance between the two word packs. 

The tekcher using the decision rules indicated that two major 
points about the decision rules wer^ important to ^coX^ider . First, 
the student appeared to perform better when, he saw tive results. 
This indicates the importance of positive feedback. Second, the 
student had a role in deciding on the nature of- his interventions; 
this (SeeiTjed to provide an incentive for. him. Additionally, the 
regularity in which changes were^ made in condition B appeared to be 
motivating for the student. However, over a prolonged period of time, 
it may Ibe difficult to generate effective interventions. In that 
case, one might opt to switch to the decision rule of condition A. 

In . conclusion, it appear^ that the decision rule operating in 
cohdillion^^B, whereby changes were introduced regularly in an attempt 
to^ enhance continuously educational programs, is ^somewhat more 
effective in producing greater- progress than the decision rule 



operating in condition A, whereby changes were introduced only after 
performance fell below an expected level for 3 consecutive days. 
However, these results need to -be investigated further with other 
students in other settings, and with other teachers. 
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